Deep Learning-Based Regression and Classification for Automatic Landmark Localization in Medical Images

In this study, we propose a fast and accurate method to automatically localize anatomical landmarks in medical images. We employ a global-to-local localization approach using fully convolutional neural networks (FCNNs). First, a global FCNN localizes multiple landmarks through the analysis of image patches, performing regression and classification simultaneously. In regression, displacement vectors pointing from the center of image patches towards landmark locations are determined. In classification, presence of landmarks of interest in the patch is established. Global landmark locations are obtained by averaging the predicted displacement vectors, where the contribution of each displacement vector is weighted by the posterior classification probability of the patch that it is pointing from. Subsequently, for each landmark localized with global localization, local analysis is performed. Specialized FCNNs refine the global landmark locations by analyzing local sub-images in a similar manner, i.e. by performing regression and classification simultaneously and combining the results. Evaluation was performed through localization of 8 anatomical landmarks in CCTA scans, 2 landmarks in olfactory MR scans, and 19 landmarks in cephalometric X-rays. We demonstrate that the method performs similarly to a second observer and is able to localize landmarks in a diverse set of medical images, differing in image modality, image dimensionality, and anatomical coverage.Comment: 12 pages, accepted at IEEE transactions in Medical Imagin

Severity of olfactory deficits is reflected in functional brain networks-An fMRI study

Even though deficits in olfactory function affect a considerable part of the population, the neuronal basis of olfactory deficits remains scarcely investigated. To achieve a better understanding of how smell loss affects neural activation patterns and functional networks, we set out to investigate patients with olfactory dysfunction using functional magnetic resonance imaging (fMRI) and olfactory stimulation. We used patients' scores on a standardized olfactory test as continuous measure of olfactory function. 48 patients (mean olfactory threshold discrimination identification (TDI) score=16.33, SD=6.4, range 6 - 28.5) were investigated. Overall, patients showed piriform cortex activation during odor stimulation compared to pure sniffing. Group independent component analysis indicated that the recruitment of three networks during odor stimulation was correlated with olfactory function: a sensory processing network (including regions such as insula, thalamus and piriform cortex), a cerebellar network and an occipital network. Interestingly, recruitment of these networks during pure sniffing was related to olfactory function as well. Our results support previous findings that sniffing alone can activate olfactory regions. Extending this, we found that the severity of olfactory deficits is related to the extent to which neural networks are recruited both during olfactory stimulation and pure sniffing. This indicates that olfactory deficits are not only reflected in changes in specific olfactory areas but also in the recruitment of occipital and cerebellar networks. These findings pave the way for future investigations on whether characteristics of these networks might be of use for the prediction of disease prognosis or of treatment success

Coming to your senses : Effects of changes in olfactory and gustatory function on eating behavior and the brain

Changes in olfactory function are a widespread problem: 3% up to 20% of the general population suffers from changes in olfactory function. Changes in gustatory function are less common: of all patients seeking clinical assistance for changes in olfactory and gustatory function, less than 4% is diagnosed with changes in gustatory function. Smell and taste, and their combined perception in flavor of food, are important determinants for food intake and subsequently nutritional status. However, the effect of changes in olfactory and gustatory function on eating behavior and the neurobiology of smell and taste is not well enough understood. Hence, within this thesis, we aimed to gain more insight in the effect of these changes on eating behavior and the neurobiology of smell and tast, in order to ultimately provide patients with sufficient health care and nutritional recommendations.In chapter 2, we used an online survey to investigate the effect of changes in olfactory function on food preferences and adherence to dietary guidelines in a population of Dutch patients with self-reported changes in olfactory function. Patients with acquired changes in olfactory function displayed the lowest preference for high-carbohydrate foods and highest preference for low-energy foods, which was similar to the preference pattern of a control group with healthy individuals. In contrast, patients with congenital anosmia showed an aberrant pattern, with a higher preference for high-fat foods. Adherence to the dietary guidelines was similar for the patients with changes in olfactory function and the reference population, but patients did show a reduced food enjoyment. Thus, changes in olfactory function seem less important for actual measures of eating behavior, such as food preferences and intake, but more relevant for changes in subsequent flavor perception and food enjoyment.Next, in chapter 3, we followed a group of patients with colorectal cancer during and after chemotherapy treatment to assess possible changes in olfactory and gustatory function and food preferences. Here, no differences in objective olfactory and gustatory function compared to control patients were shown. However, during treatment, subjective olfactory and gustatory function were rated significantly worse by patients undergoing chemotherapy than the control patients. We found no effect of undergoing chemotherapy treatment on food preferences, but preference for protein was positively correlated with objective gustatory function. This correlation is presumably related to the change in flavor perception that is likely to occur in patients experiencing changes in gustatory function.Chapter 4 describes the application of convolution neural networks to allow automated volume measurements of olfactory bulb volume. The olfactory bulb is the first receptor of olfactory signals in the human brain and therefore of importance to study in the context of changes in olfactory function. Localization and segmentation of the olfactory bulb as well as subsequent calculation of olfactory bulb volume were performed successfully. This method can be utilized in both research and health care and may lead to more insight in the role of the olfactory bulb in diagnosis, prognosis and treatment of olfactory lossIn chapter 5, we investigated the morphology of primary and secondary olfactory-related brain regions of patients with primary changes in olfactory function. Patients with congenital anosmia showed reduced density in the gyrus rectus compared to patients with acquired changes in olfactory function, while the density of the orbitofrontal cortex of patients with congenital anosmia was increased compared to the other patients groups. Moreover, there was a positive relation between density of the orbitofrontal cortex and olfactory function. This brain region is related to flavor perception and is known to play a role in the perceived pleasantness of foods during consumptionLastly, in chapter 6, we aimed to determine how changes in olfactory functioning affect neural activation patterns and networks in the olfactory system of the brain and how this is related to olfactory function. We found that patients with olfactory loss showed piriform cortex activation during odor stimulation compared to pure sniffing, even in anosmic patients. Moreover, olfactory function was correlated with the recruitment of a sensory processing network.This network involved several regions that are known to play a role in flavor perception, like the insula and the anterior cingulate cortex.From this thesis, we conclude that changes in olfactory and gustatory function seem less important for actual measures of eating behavior, such as food preferences and intake, but more relevant for changes in subsequent flavor perception and food enjoyment. Moreover we found that changes on morphology and function of olfactory-related brain regions were dependent on olfactory function. These changes were reflected in several brain regions that are known to play a role in flavor perception and in the perceived pleasantness of foods during consumption. Interventions for patients with changes in olfactory and gustatory function should therefore include nutritional recommendations and take food enjoyment in consideration. Moreover, health care providers should consider the role that the neurobiology of olfaction, like volume of the olfactory bulb, can play in diagnostics and prognostics of disease. Future research should focus on more longitudinal studies combining several outcome measures of eating behavior and the neurobiology of smell, taste and their integration in flavor perception. This will lead to further insights in treatment possibilities that target the neuroplasticity of olfactory-related brain regions and effective strategies in regard to eating behavior in patients with changes in olfactory and gustatory function

Smakelijk eten voor patiënten met kanker : De invloed van reuk- en smaakverstoring op eetgedrag

This article provides an overview of smell and taste changes in cancer patients and the influence of these changes on eating behaviour and food preferences. • Taste changes seem to be more common than alterations in the sense of smell. • The changes in taste and smell are often temporary; smell and taste functions mostly return to their previous level after treatment cessation. • The type of cancer and its treatment influence changes in the sense of smell and taste, making it challenging to give general advice. • Changes in taste and smell - which can lead to modified food preferences and decreased appetite - should be taken into consideration during the treatment of cancer patients to improve the nutritional status and the effectiveness of the treatment. • This requires a personal approach, a multidisciplinary team and validated methods. Moreover, it is important to involve the environment of the patient

Automatic segmentation of the olfactory bulbs in MRI

A decrease in volume of the olfactory bulbs is an early marker for neurodegenerative diseases, such as Parkinson's and Alzheimer's disease. Recently, asymmetric volumes of olfactory bulbs present in postmortem MRIs of COVID-19 patients indicate that the olfactory bulbs might play an important role in the entrance of the disease in the central nervous system. Hence, volumetric assessment of the olfactory bulbs can be valuable for various conditions. Given that manual annotation of the olfactory bulbs in MRI to determine their volume is tedious, we propose a method for their automatic segmentation. To mitigate the class imbalance caused by the small volume of the olfactory bulbs, we first localize the center of each olfactory bulb in a scan using convolutional neural networks (CNNs). We use these center locations to extract a bounding box containing both olfactory bulbs. Subsequently, the slices present in the bounding box are analyzed by a segmentation CNN that classifies each voxel as left olfactory bulb, right olfactory bulb, or background. The method achieved median (IQR) Dice coefficients of 0.84 (0.08) and 0.83 (0.08), and Average Symmetrical Surface Distances of 0.12 (0.08) and 0.13 (0.08) mm for the left and the right olfactory bulb, respectively. Wilcoxon Signed Rank tests showed no significant difference between the volumes computed from the reference annotation and the automatic segmentations. Analysis took only 0.20 second per scan and the results indicate that the proposed method could be a first step towards large-scale studies analyzing pathology and morphology of the olfactory bulbs. </p

Morphological changes in secondary, but not primary, sensory cortex in individuals with life-long olfactory sensory deprivation

Individuals with congenital sensory deprivation usually demonstrate altered brain morphology in areas associated with early processing of the absent sense. Here, we aimed to establish whether this also applies to individuals born without a sense of smell (congenital anosmia) by comparing cerebral morphology between 33 individuals with isolated congenital anosmia and matched controls. We detected no morphological alterations in the primary olfactory (piriform) cortex. However, individuals with anosmia demonstrated gray matter volume atrophy in bilateral olfactory sulci, explained by decreased cortical area, curvature, and sulcus depth. They further demonstrated increased gray matter volume and cortical thickness in the medial orbital gyri; regions closely associated with olfactory processing, sensory integration, and value-coding. Our results suggest that a lifelong absence of sensory input does not necessarily lead to morphological alterations in primary sensory cortex and extend previous findings with divergent morphological alterations in bilateral orbitofrontal cortex, indicating influences of different developmental processes

Seeing Beyond Your Nose? The Effects of Lifelong Olfactory Sensory Deprivation on Cerebral Audio-visual Integration

Lifelong auditory and visual sensory deprivation have been demonstrated to alter both perceptual acuity and the neural processing of remaining senses. Recently, it was demonstrated that individuals with anosmia, i.e. complete olfactory sensory deprivation, displayed enhanced multisensory integration performance. Whether this ability is due to a reorganization of olfactory processing regions to focus on cross-modal multisensory information or whether it is due to enhanced processing within multisensory integration regions is not known. To dissociate these two outcomes, we investigated the neural processing of dynamic audio-visual stimuli in individuals with congenital anosmia and matched controls (both groups, n = 33) using functional magnetic resonance imaging. Specifically, we assessed whether the previously demonstrated multisensory enhancement is related to cross-modal processing of multisensory stimuli in olfactory associated regions, the piriform and olfactory orbitofrontal cortices, or enhanced multisensory processing in established multisensory integration regions, the superior temporal and intraparietal sulci. No significant group differences were found in the a priori hypothesized regions using region of interest analyses. However, exploratory whole-brain analysis suggested higher activation related to multisensory integration within the posterior superior temporal sulcus, in close proximity to the multisensory region of interest, in individuals with congenital anosmia. No group differences were demonstrated in olfactory associated regions. Although results were outside our hypothesized regions, combined, they tentatively suggest that enhanced processing of audio-visual stimuli in individuals with congenital anosmia may be mediated by multisensory, and not primary sensory, cerebral regions

Anosmia-A Clinical Review.

Anosmia and hyposmia, the inability or decreased ability to smell, is estimated to afflict 3-20% of the population. Risk of olfactory dysfunction increases with old age and may also result from chronic sinonasal diseases, severe head trauma, and upper respiratory infections, or neurodegenerative diseases. These disorders impair the ability to sense warning odors in foods and the environment, as well as hinder the quality of life related to social interactions, eating, and feelings of well-being. This article reports and extends on a clinical update commencing at the 2016 Association for Chemoreception Sciences annual meeting. Included were reports from: a patient perspective on losing the sense of smell with information on Fifth Sense, a nonprofit advocacy organization for patients with olfactory disorders; an otolaryngologist's review of clinical evaluation, diagnosis, and management/treatment of anosmia; and researchers' review of recent advances in potential anosmia treatments from fundamental science, in animal, cellular, or genetic models. As limited evidence-based treatments exist for anosmia, dissemination of information on anosmia-related health risks is needed. This could include feasible and useful screening measures for olfactory dysfunction, appropriate clinical evaluation, and patient counseling to avoid harm as well as manage health and quality of life with anosmia

Automatic segmentation of the olfactory bulbs in MRI

A decrease in volume of the olfactory bulbs is an early marker for neurodegenerative diseases, such as Parkinson's and Alzheimer's disease. Recently, asymmetric volumes of olfactory bulbs present in postmortem MRIs of COVID-19 patients indicate that the olfactory bulbs might play an important role in the entrance of the disease in the central nervous system. Hence, volumetric assessment of the olfactory bulbs can be valuable for various conditions. Given that manual annotation of the olfactory bulbs in MRI to determine their volume is tedious, we propose a method for their automatic segmentation. To mitigate the class imbalance caused by the small volume of the olfactory bulbs, we first localize the center of each olfactory bulb in a scan using convolutional neural networks (CNNs). We use these center locations to extract a bounding box containing both olfactory bulbs. Subsequently, the slices present in the bounding box are analyzed by a segmentation CNN that classifies each voxel as left olfactory bulb, right olfactory bulb, or background. The method achieved median (IQR) Dice coefficients of 0.84 (0.08) and 0.83 (0.08), and Average Symmetrical Surface Distances of 0.12 (0.08) and 0.13 (0.08) mm for the left and the right olfactory bulb, respectively. Wilcoxon Signed Rank tests showed no significant difference between the volumes computed from the reference annotation and the automatic segmentations. Analysis took only 0.20 second per scan and the results indicate that the proposed method could be a first step towards large-scale studies analyzing pathology and morphology of the olfactory bulbs

Normal Olfactory Functional Connectivity Despite Lifelong Absence of Olfactory Experiences

Congenital blindness is associated with atypical morphology and functional connectivity within and from visual cortical regions; changes that are hypothesized to originate from a lifelong absence of visual input and could be regarded as a general (re) organization principle of sensory cortices. Challenging this is the fact that individuals with congenital anosmia (lifelong olfactory sensory loss) display little to no morphological changes in the primary olfactory cortex. To determine whether olfactory input from birth is essential to establish and maintain normal functional connectivity in olfactory processing regions, akin to the visual system, we assessed differences in functional connectivity within the olfactory cortex between individuals with congenital anosmia (n = 33) and matched controls (n = 33). Specifically, we assessed differences in connectivity between core olfactory processing regions as well as differences in regional homogeneity and homotopic connectivity within the primary olfactory cortex. In contrast to congenital blindness, none of the analyses indicated atypical connectivity in individuals with congenital anosmia. In fact, post-hoc Bayesian analysis provided support for an absence of group differences. These results suggest that a lifelong absence of olfactory experience has a limited impact on the functional connectivity in the olfactory cortex, a finding that indicates a clear difference between sensory modalities in how sensory cortical regions develop.</p